Compression of digital video data is used for many applications including transmission over bandwidth-constrained channels, such as direct broadcast satellite, and storage on optical media. In order to achieve very efficient compression, complex, computationally intensive processes are used for encoding (compressing) and decoding (decompressing) video. For example, although MPEG-2 (Moving Pictures Expert Group, International organization for Standards, Geneva, Switzerland) is known as a very efficient method for compressing video, a new, more efficient standard, H.264 (“Advanced Video Coding”, International Telecommunication Union Telecommunication Standardization Sector, Geneva, Switzerland), is being developed.
The H.264 standard allows for bitstreams that (1) use a large number of reference frames to reconstruct a single picture and (2) use reordering schemes that transmit many “future frames” with later display times then a current picture before the current picture is transmitted. By contrast, MPEG-1 and MPEG-2 allow for at most two reference frames for reconstructing a picture and only a single future frame.
Referring to FIG. 1, a diagram illustrating a conventional bitstream 10 that uses many reference frames is shown. In the illustration, a group of pictures (frames) having six pictures is represented by one I-frame 12 followed by five P-frames 14a-e. Each P-frame 14a-e after the I-frame 12 uses all of the previous frames in the group of pictures as references, so that the last P-frame 14e has five reference frames 12 and 14a-d. 
Referring to FIG. 2, a diagram illustrating a conventional bitstream 16 that uses many “future frames” is shown. The bitstream 16 is shown in a display order with arrows indicating how reference frames are used for prediction. Like groups of pictures often used for MPEG-1 and MPEG-2, each P-frame 18a-f is predicted from one I-frame 20 or P-frame 18a-f, and each B-frame 22a-l is predicted from two frames, each of which is an I-frame 20 or a P-frame 18a-f. However, the H.264 standard allows the I-frame 20 to be displayed in the middle of the group of pictures. The frames 18a-c and 22a-f that are displayed before the I-frame 20 are predicted in the opposite direction as usual, that is the P-frames 18a-c before the I-frame 20 use backward only, instead of forward only, prediction. Thus, the I-frame 20 is transmitted first but is displayed tenth. Depending on the transmission order of the B-frames 22a-l (i.e., if the B-frames 22a-f before the I-frame 20 are transmitted in display order, backwards display order, or something else), the decoder will need to buffer five to ten frames to decode and display the bitstream 16.
The flexible approach permitted by the H.264 standard for creating bitstreams results in the decoder buffering a large amount of image data. To limit the amount of memory that a decoder reasonably uses for decoding, the H.264 standard places two constraints on the bitstreams (i) a bitstream cannot be constructed so that the total number of bytes of decompressed pictures buffered at the decoder exceeds a limit B and (ii) a bitstream cannot be constructed so that the total number of decompressed frames buffered at the decoder exceeds a limit F. For example, for a level 4 (high definition) stream, at any time, the decoder will never hold more than sixteen frames or any number of frames that use more than 12,288×1024 bytes in total.
The H.264 standard allows for a resolution of compressed pictures in a single bitstream to change. Thus, the maximum number of frames buffered over time will vary with the resolution of the frames. When low-resolution frames are used, many frames are commonly buffered. When high-resolution frames are used, few frames are commonly buffered.
If memory is accessed through register read-write instructions using virtual linear memory, the total amount of memory that a decoder will allocate for decoded pictures is roughly the nominal limit given by the H.264 standard (i.e., 12,288 Kbytes for level 4). In practice an actual decoder uses slightly more memory when not operating exactly according to the principles of the H.264 reference decoder. For example, an extra delay between decoding and display introduced for scaling or other display processing consumes additional memory.
A conventional video decoder allocates physically contiguous buffers for the maximum number of frames used for decoding times the maximum size of each frame. The buffers are used for storing frames as the frames are decoded and freed when the frames are no longer needed (i.e., the frames that have been displayed and will no longer be used as references for other frames). The conventional approach will use much more memory than a nominal amount of memory if the number of frames to be buffered depends on the resolution of the frames.